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ABSTRACT 

Accommodating the planetarium experience to members of the deaf or hard-of-hearing community 
has often created situations that are either disruptive to the rest of the audience or provide an 
insufficient accommodation. To address this issue, we examined the use of head-mounted displays 
to deliver an American Sign Language “sound track ” to learners in the planetarium. ” Here we 
present results from a feasibility study to see if an ASL “sound track” delivered through a head- 
mount display can be understood by deaf junior to senior high aged students who are fluent in 
ASL. We examined the adoption of ASL classifiers that were used as part of the “sound track" for 
a full dome planetarium show. We found that about 90% of all students in our sample adopted at 
least one classifier from the show. In addition, those who viewed the “sound track” in a head- 
mounted display did at least as well as those who saw the “sound track”projected directly on the 
dome. These results suggest that ASL transmitted through head-mounted displays is a promising 
method to help improve learning for those whose primary language is ASL and merits further 
investigation. 
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he modern planetarium depicts an immersive environment of sights and sounds that can captivate an 
audience. Many full dome shows can be fast-paced with interesting visuals on all parts of the dome. 
Even in a traditional live planetarium show, the pace can be accelerated with the presenter jumping to 
visuals on various parts of the dome. While this can be an exciting, informal learning environment for many people, 
it also presents challenges for some audience members. For audience members who are deaf or hard-of-hearing and 
who communicate primarily in American Sign Language (ASL), the challenges of a planetarium environment are 
most often prohibitive. Two traditional approaches to accommodating a deaf audience member exist; 1) Put an ASL 
signer on the floor and illuminate him/her with a red light, or 2) use some type of captioning system. The former of 
these approaches has three impacts on the audience: 1) There is light splash on the dome surface, 2) distraction to 
the hearing audience, and 3) split attention is needed to look back and forth between the signer and the dome. 
Beyond just an interference, a captioning system can also cause a split attention issue and a distraction for the 
remainder of the audience (Moreno & Mayer, 2002). More importantly, captions may not be effective for deaf 
children as reading comprehension scores for deaf children are significantly lower than scores of their hearing peers 
(Qi & Mitchell, 2012). In this work we offer a potential third accommodation method by delivering an ASL “sound 
track” to a deaf audience that removes distractions to a hearing audience, limits split-attention issues, and removes 
the need for significant English reading skills. 


We have established a research effort to test the potential of delivering an ASL “sound track” to audience 
members using a head-mounted display (HMD). Figure 1 illustrates the basic idea of this system for three children 
in a planetarium show. Child 1 and child 2 wear HMDs that allow them to see an ASL signer regardless of where 
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they are looking in the dome. Child 3 is a hearing child who does not wear an HMD but sees and hears the show as 
normal. Child 3 is ideally unaware of the accommodation being made for child 1 and child 2. It is important that the 
accommodation for the first two children does not impact the learning of the third child. In a first stage of testing, 
Jones et al. (2014) examined the technology side of the issue to see if it was even possible to deliver ASL through an 
HMD system. In that paper they addressed two important technical questions. First, when are HMDs usable for 
viewing sign language by fluent ASL users? And secondly, when are HMDs comfortable for viewing sign language 
by fluent ASL users? They discussed three different HMD systems that were tested, as well as the preference of 
ASL signer location. However, the testing by Jones et al. (2014) was executed in a normal room with visuals in a 
single direction. The subjects for the original testing were drawn from students attending a summer program who 
had come from around the country. In subsequent years that population of students was not made available for 
further studies. This severely limited the potential population of test subjects for this study. 




Figure 1 . Three children at a planetarium show. Two deaf (or hard-of-hearing) children wear HMDs and see a sign language 
narration in different places. The sign language narration is not visible to the third hearing child who listens to a normal audio 
narration. 


Building on the work reported in Jones et al. (2014), we wanted to examine the use of the HMDs in the full 
planetarium environment. However, the ability to recruit a significant number of deaf students within a reasonably 
short travel distance was very limited. Therefore, this second phase of testing was designed as a feasibility study to 
see if it would be worth the time, expense, and travel necessary to enact a larger study designed to truly test learning 
gains with HMDs. The question posed here is, can junior to senior high aged deaf students, whose primary language 
is ASL, watch a full dome planetarium show with an ASL “sound track” and understand new terminology shown to 
them in said “sound track”? If students could see and then mimic ASL classifiers used to describe astronomical 
objects as part of a full dome planetarium show, then compiling a much larger study, which would require extensive 
travel in order to recruit a larger sample, would be beneficial. The purpose of this work was not to show learning, 
but to demonstrate that the students could see the “sound track” and the full dome graphics at the same time. This 
has an impact on future tests for learning and the potential of HMDs as appropriate accommodations for deaf 
audience members. 


Since a majority of people reading this paper may not have extensive experience with the issues facing deaf 
audience members, we will first give a general overview of education research with deaf students and the issues 
those students face in various learning environments. Because a second important issue for the HMD systems is 
their potential as an accommodation method in planetarium shows, we will discuss the history of accommodation 
methods used in planetariums and how those results support the future use of new technology like HMDs. We will 
then detail the experimental design used in our testing. Finally, we will examine the results of this feasibility study 
on a small group of deaf students to determine if further work is worth pursuing. 
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BACKGROUND AND CONTEXT 
Deaf Children’s Educational Experience 

We must start by stating that the work presented here centers on the deaf educational experience of students 
only in the United States. Deaf children in other countries encounter different sets of opportunities and challenges. 
Lytle et al. (2005) studied deaf education in China and described a different set of cultural and educational 
challenges and opportunities faced by children in that country. A thorough discussion of how deaf children learn can 
be found in Marschark and Hauser’s (2011) lucid evidence-based survey. Nevertheless, much remains unknown 
concerning how deaf and hard-of-hearing children learn, and even less about how deaf children (or adults) 
experience sign language in HMDs. However, many studies have been performed on their language and reading 
abilities. 


Over 30 years of educational testing at grade levels K-12 (5 to 17-year-old children) in the United States 
show the average reading comprehension score for deaf children is below the average comprehension score for 
fourth grade (9-year-old) hearing readers taking the same test (Qi & Mitchell, 2012). While well-intentioned but 
unsuitable accommodations made to deaf children when administering written standardized tests may explain some 
of this persistent discrepancy, these results suggest that adding captions may not improve learning for deaf audience 
members. Facilitating the delivery of sign language should improve comprehension and in turn improve reading 
skills. Chamberlain and Mayberry (2000) provided an overview, and Hermans, Knoors, Ormel, and Verhoeven 
(2008) conducted a more recent study in this topic. This leads to the driving principle behind our work, that 
delivering information in sign language rather than in written captions is particularly important for children who 
have been learning sign language as their first language. 

Compared to spoken language acquisition by children who hear, children who are deaf or hard-of-hearing 
often experiences significantly delayed acquisition of their first language (Marschark & Hauser, 2011). Additionally, 
young deaf and hard-of-hearing children often do not receive full linguistic input every day. Two factors likely 
contribute to this delay. First, if a child is not identified as deaf or hard-of-hearing until two years of age, for 
example, that child will have missed critical years of language input. Second, even when a child is identified as deaf 
or hard-of-hearing, a hearing parent with no knowledge of sign language would require a significant amount of time 
to become proficient in sign language. It is estimated that 95% of school-age deaf and hard-of-hearing children are 
born to hearing parents (Mitchell & Karchmer, 2004). These children often do not begin learning sign language until 
entering school and may only be receiving fluent language input during school hours. The period from birth to age 
two is a critical time for the acquisition of language and cognition for all children, and this period of time is often 
when deaf and hard-of-hearing children are deprived of processes that promote healthy language development 
(Humphries et al., 2012). 

During this period of early life, many deaf and hard-of-hearing children are sometimes unintentionally and 
unknowingly unable to access the language of their families or peers because this language is not in a visual form. In 
the absence of a visual language such as ASL, the risk of harm from language deprivation is heightened and their 
cognitive capacities become reduced. Language deprivation occurs when a child does not receive sufficient 
language input to acquire or learn any language or readily develop cognitive capabilities. The presence of a signed 
language from birth greatly reduces this risk of harm (Humphries et al., 2012). 

As mentioned previously, improved sign language learning may lead to improved learning of a second 
language such as English. Research shows “that children who learn through their first (minority) language for as 
long as possible not only tend to have improved final achievement, but also their English language skills tend to 
develop to a higher level than those who were taught through their second language with some first language 
support” (Knight & Swanwick, 2002). Unfortunately, strict curriculum standards in many parts of the United States 
require students to begin reading and writing in Kindergarten. Teachers cannot purely use sign language in the 
classroom and wait until children develop their first language before introducing English. If children do not fully 
develop a first language, they cannot fully develop their second language, and therefore, they become limited 
bilinguals (Baker, 2011). 
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Improved signed and written language abilities may improve STEM education outcomes. A child has to 
have basic language before he or she can comprehend more complex words, such as “force,” “hypothesis,” 
“estimate,” “solve,” and so forth. Even if a teacher can provide the students with visual examples and hands-on 
experiences relating to the scientific vocabulary, students often forget the vocabulary due to both a lack of repeated 
exposure and a lack of true understanding of the vocabulary. 

Visual Attention and Cognition 

A human factors approach or applied attention theory directly relates to the work presented here. Perhaps 
the easiest way to understand the idea is to think of an airplane pilot whose primary task is to fly a plane. The pilot 
is held responsible for flying the plane but must gather information from his/her many instmments in order to 
accomplish that task. This will likely require eye and head movements, along with search times, to find the right 
piece of information. Clearly the overall task requires splitting, or switching, one’s visual attention between multiple 
input sources so that the main task can be completed successfully. A good summary of these principles can be 
found in Wicken and McCarley (2009). Interestingly, according to Marschark and Hauser (2011), deaf children are 
more adept at central visual and peripheral visual field switching than hearing children. This ability to shift their 
visual attention faster may play an important role when that shift is shortened by the use of an HMD. 

The research of Sweller, Ayers, and Kalyuga (2011) on the split-attention effect has been quite extensive. 
They found that people learn less effectively when attention splits between at least two sources of information that 
are separated either spatially or temporally. This split in attention causes an increase in cognitive load when short¬ 
term memory becomes split over two to three items of simultaneous information (Kirschner, 2002). The split- 
attention effect, as defined by Sweller et al. (2011), is seen when learning increases due to placing information 
sources in close proximity. The split-attention effect has been investigated in a number of studies that suggest the 
need to move information sources together to decrease cognitive load (Sweller et al., 2011; Kirschner, 2002; 
Chandler & Sweller, 1991). A specific example of how combining multimedia materials to reinforce related 
information increases learning can be found in Kirschner (2002). For example, if the legend for a bar graph, which 
contains the meaning of each bar in the graph, is shown at the bottom of the graph, then visual information is split 
spatially across the graph. This spatial split increases cognitive load and reduces understanding. However, if 
information were placed closer together by placing labels directly on each bar, then cognitive load is reduced and 
learning increased. Furthermore, cognitive load does not favor one type of information over another (Kirschner, 
2002 ). 

In our work we attempt to maximize the split-attention effect by placing the signer in close proximity to 
other visual cues provided as part of the planetarium show. The HMD allows the signer to remain in the child’s field 
of view even when the child turns his/her head to follow the graphics. On a long-term basis, we endeavor to 
understand how to utilize this effect appropriately with the use of ASL on HMDs in logistic ally challenging 
environments, such as large lecture halls, labs, planetariums, and so forth. 

Research Using Related Technology 

The use of HMD for the deaf or hard-of-hearing is not unique to this project. Kaufmann and Schmalstieg 
(2003) used a stereoscopic HMD to teach children mathematics and geometry through 3D interaction of 3D 
augmented reality (AR) objects. The subjects used Construct3D, a program specifically designed for the 3D AR 
system called “Studierstube.” Studierstube allows multiple users to interact in the same 3D AR environments. 
Kaufmann found that most participants saw the potential of such a system to learn geometry in a new 3D way 
(Kaufmann & Schmalstieg, 2003). However, a few students felt ill after their session and may have experienced 
what the authors called ‘cybersickness. ’ 

Similarly, Jimenez, Iglesias, Lopez, Hernandez, and Ruiz (2011) compared the use of HMDs and tablet 
computers using a closed and open-captioned system in a classroom setting. The closed and open-captioned systems 
involved real-time translation of spoken text into captions. All subjects were over the age of 16, but it is not clear if 
any subjects were deaf or hard-of-hearing. They reported no significant preference between using an HMD and a 
tablet computer. 
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We are exploring multimedia learning in the sense that we study learning using “words and pictures” (to 
use the Mayer [2009] definitions). However, in our case the words are formed using signs rather than sound or 
written letters. Our work builds on current understanding of visual attention and cognition and on prior work in the 
education of children who are deaf or hard-of-hearing. 

Accommodation in a Planetarium 

Captioning Systems in a Planetarium 

Beyond the educational questions that can be addressed using the HMD systems, the issue of 
accommodations for deaf or hard-of-hearing students in a planetarium, or in other challenging environments, also 
exists. The HMD potentially provides the least intrusive accommodation method on the audience at large. Possibly, 
the majority of a planetarium show audience would not be aware of an accommodation made since no visible 
difference subsists for the hearing portion of the audience. With modern wireless systems, HMDs could also allow 
deaf audience members to sit anywhere in a planetarium versus being only in assigned seats. It was not uncommon 
to hear participants in this study say they could now sit wherever they liked and in some sense be just like every 
other audience member. With that in mind, we want to explore the nature of historical planetarium accommodations 
as they relate to our work. 

Several groups have explored methods for delivering captions in a planetarium setting. Perhaps one of the 
early, more well-documented efforts concerning delivering captions in this setting is reported by DeGraff and Hamil 
(1972). After a first experience with a group of deaf individuals that came to a planetarium with interpreters, they 
realized that students gained limited understanding. Their second attempt was to have interpreters give a 15-minute 
pre-show discussion. This change improved comprehension but broke up the continuity of the show. At this point 
they moved to the creation of captions to be projected with a slide projector. They quickly realized a need to cut the 
number of words down from the script to the captions, in most cases using less than half the original words. In this 
version the captions were projected low in the planetarium. Daniel (1974) details a slightly different way of using 
captions for a live show. In this case “word slides” were created. The “word slides” were either projected near the 
object of interest on the dome, or a green arrow was used to move the student’s eyes to the right area of the dome, an 
interesting method given the modern studies of split-attention discussed earlier. 

Shea (1993) detailed a system installed in 1988 to project captions below the dome level in a planetarium to 
provide a “sound track” for a pre-recorded planetarium show to a deaf audience. She also pointed out that this 
system can be valuable for anyone with hearing limitations. Here we list a few of the important findings from Shea 
(1993): (1) The captions had to be simplified due to potential vocabulary limitations for some audience members, 
(2) they needed to pace new shows slower to give visitors time to read, (3) the hearing impaired people would have 
preferred the captions on the dome near the visuals, (4) hearing audience members had no adverse response, and (5) 
although groups appreciated the effort, the response was not overwhelming. Interestingly, the request in finding 
number three is very similar to the technique used in Daniel (1974) and is supported by the multimedia testing of 
Mayer (2008) and thus constitutes a significant component of the work presented in this paper. 

In Grice (1996) we find the beginnings of more modern approaches to captioning systems, a major move 
into devices for planetariums and theaters. Grice (1996) reported on attending a demonstration of three “new” 
technologies held in Boston in 1993. The first device tested was a pair of Virtual Vision glasses that showed 
captions over the right eye in a small screen. Reportedly, most people put these away after a few minutes and 
reported a “dizzying effect.” The second accommodation was an LED display system that was mounted behind the 
audience. This displayed the captions in reversed text, and Plexiglas was mounted on specific seats that would then 
reflect the captions to the individual. However, this system could not be installed in a concentric seat configuration, 
typical of many planetariums. The final test included a Vacuum Fluorescent Display (VFD), a box that attached to a 
seat in front of the individual that had captions run across the screen. The planetarium installed four VFD captioning 
systems that could each support three people. Grice (1996) reported as having had success with this system. 
However, this captioning system still has all the issues detailed earlier concerning children’s reading 
comprehension. It should also be noted that this device limits the seating choices for those audience members using 
the accommodation method and does not address the split-attention issue since the captions are not near the visual. 
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The use of HMDs has the potential to provide a much richer planetarium experience for a deaf audience. In 
most cases the captioning systems either require very fast reading skills or have simplified captions to improve the 
speed. Although modern HMDs are better than those reported in Grice (1996) and could be used to provide caption 
near the objects of interest, we still have the issue of reading speed and comprehension, especially for younger 
audiences. With a full ASL translation that can be viewed at the same time as the full dome video, the deaf audience 
has the potential to obtain as much out of the show as the hearing audience. However, the question still remains: Can 
understandable ASL be delivered in an HMD at sufficient speed in a full planetarium show? 

METHOD AND RESULTS 


ASL Classifiers 

Taking into account all the information detailed above, we set to work developing a system to deliver ASL 
to an HMD system. The results presented in Jones et al. (2014) detailed the technical issues of putting an ASL signer 
into an HMD system that was comfortable enough to wear and allowed the head to move in all directions to provide 
spatial coverage. That research also discussed the preference for image location being centered over the eye and not 
in the peripheral vision areas. Once the basic technology issues were addressed, we were ready to test the ability of 
HMDs to deliver ASL that could be understood by those fluent in ASL. 

The question that arises at this point is, can someone watching a full dome planetarium show while wearing 
an HMD see the ASL “sound track” well enough to be able to recognize the symbols and mimic new combinations? 
For our testing we had the advantage because the majority of astronomical objects do not have official ASL signs. 
There are a couple options for conveying information when an official ASL sign does not exist: Either the words can 
be finger spelled each time, or a set of classifiers can be used to describe the object. Clearly finger spelling is just a 
version of English and would act much like captions in delivering information. We felt the better approach would 
be to use ASL classifiers. 

An ASL classifier is a handshape that conveys size, shape, usage, or perhaps motion. Classifiers are 
recognized symbols that would be part of any fluent ASL person’s “vocabulary” and would be known by our test 
subjects. An example might be a closed fist representing a solid sphere, or spreading out five fingers to represent 
light being emitted. Combining the two classifiers mentioned above with the motion of the hands, one can describe 
a comet (as shown in Figure 2)—one hand as a closed fist (solid sphere) with a spreading of five fingers coming off 
(light or particle emission) with the two hands moving (motion of the object). For speed and understanding, the use 
of classifiers remains preferable to finger spelling. 

Even though the students had a previous knowledge of the individual classifiers as part of their ASL 
vocabulary, they likely have never seen the classifiers combined to describe a specific astronomical object. 
Therefore, the classifiers made an ideal measurement instrument in our testing. By testing before the show, we could 
see if any of the students would use specific classifiers to describe the objects on the pre-test interview. Then the 
students would watch the planetarium show where the classifiers were combined to describe objects or events. If the 
students then used the classifiers in these new combinations during post testing to describe the object or event, then 
we would know that they had seen the classifier as part of the show. It is important to note that those who 
interviewed the test subjects were instructed not to use the classifiers from the show. This ensured that the test 
subjects were using the classifiers because they had seen them in the show. 
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Figure 2. Combination of Two Classifiers for a Solid Sphere and the Emission of Light, Combine with Motion to Represent a 
Comet. 


To begin preparation for testing, we obtained a full English transcript from Evans & Sutherland for its 
digitally produced planetarium show called “New Horizons.” The script was given to a Certified Deaf Interpreter 
(CDI) so that the narration of the show could be translated into an ASL recording. This required the interpreter to 
use combinations of ASL classifiers to convey meaning to astronomical terms presented in the show that had no 
ASL equivalent. The CDI performed the translation in front of a green screen while wearing a light blue shirt. The 
green screen was replaced with black background. Black appears transparent in a see-through HMD or when 
projected onto the dome surface. 

HMD Displays Used in Testing 

The three displays used in our evaluations are shown for comparison in Figure 3, with specifications given 
in Table 1. We used only monocular displays to maximize the amount of light reaching the eye because adding 
stereoscopic 3D to the display does not necessarily improve comprehension when viewing sign language. The 
display on the left of Figure 3 is a fully occlusive Virtual Realities VR1. The center display is a Vusix Tac-eye 
partially occlusive display. The display on the right is a Laster PMD-G2 see-through display with a half-silvered 
mirror that blocks some incoming light. Table 1 contains the diagonal viewing angle, resolution, color depth, and 
weight for each display. Viewing sign language at a resolution of 800x600 is not likely to negatively impact 
comprehension (Weaver, Starner, & Hamilton, 2010). Weaver et al. (2010) reported that novice signers could 
observe and reproduce specific signs with equal success when learning those signs from video rendered at 640x480, 
320x240, or 160x120 on a mobile phone screen. We need to report that all three displays listed in Table 1 were 
large, bulky, and heavy for use by children. We hope that future development of HMDs will make them smaller, 
lighter, and perhaps designed specifically for children. 
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Figure 3. The three HDMs used in evaluations. The displays are all monocular and include fully occlusive on the left, partially 
occlusive in the middle, and see-through with a half-silvered mirror on the right. Monocular displays were used to maximize the 
amount of light reaching the eye. 


Table 1. Characteristics of Head-Mounted Displays 


Display 

Field of View 

Resolution 

Color 

Weight 

Virt. Realities VR1 

o 

o 

800x600 

24 bits 

3 oz. 

Vusix Tac-eye 

29.5° 

800x600 

8 bits 

1.8 oz. 

Laster PMD-G3 

50° 

800x600 

24 bits 

3 oz. 


Method of Planetarium Testing 

Because the subjects in this study communicate primarily in ASL, care was taken to minimize linguistic 
barriers. Interactions with the subjects entailed direct ASL-to-ASL communication without any intervening 
interpreter. The investigating team includes deaf and hard-of-hearing individuals who use ASL as their primary 
language. Deaf and hard-of-hearing individuals who use ASL as their native language, but who are not part of the 
investigation team, were recruited to interact with and interview subjects in order to avoid bias. Interpreters 
facilitated communication between ASL-speaking and English-speaking investigators but did not facilitate 
interaction with the subjects. This kept potential communication problems due to the language barrier at the level of 
staff interactions. 

Testing for this phase of the project was done in the Royden G. Derrick Planetarium at Brigham Young 
University. The planetarium has a 39’ diameter dome and is equipped with an E&S Digistar5 full dome video 
system, along with all appropriate theater controls. With 119 seats there was plenty of room for test subjects, 
researchers, and equipment. As a first phase of testing, we conducted a small pilot study with a group of three deaf 
children from a single family. Figure 4 shows those three children in the planetarium during testing. This group was 
used primarily to test set-up procedures for the HMDs, positioning of subjects within the planetarium, and to check 
timing issues. The family group watched the planetarium show, with the children viewing the ASL narration track 
through the HMDs. They were interviewed afterwards about the experience but were not given pre- and post-tests. 
The children were excited and positive about the experience. This gave us the confidence to proceed to the more 
extensive testing phase. 

For the primary testing phase, we were only interested in the impact of projected ASL interpreters, either 
directly on the dome or through HMDs. The option of displaying the ASL “sound track” on the dome might be an 
option for a purely deaf audience but would not be appropriate for a mixed hearing and deaf audience and is not 
common practice. It should be noted that the ASL track projected directly on the dome was much larger, and 
therefore, it would theoretically be easier to see the classifiers. In the end we wanted to see if using HMDs would be 
just as effective for students to see and understand the usage of the classifiers as having the ASL track displayed 
directly on the dome. 

Our group of test subjects came from a nearby charter school for the deaf. This group was comprised of 17 
junior and senior high aged students, represented the entire testing pool from the school, and accounted for the 
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majority of test subjects within a reasonable distance from the planetarium. This explains the small sample size and 
why a larger study could not be done at this point. The students were randomly assigned to one of two treatment 
groups by drawing a badge from a sack. The first set of the students would view the ASL narration projected directly 
onto the dome of the planetarium. A second set of students would watch the show with the ASL narration being 
displayed in an HMD. The nine students who saw the narration on the dome surface are collected on the right side of 
Table 2 with the number of times they used classifiers on the pre-test and post-test interviews listed. The same 
information for the eight students who used the HMDs is given on the left side of Table 2. 



Figure 4. Subjects viewed video projected onto a planetarium dome while watching ASL narration either in the HMD, as shown, 
or projected directly onto the dome. Facilitators seated to the right of each subject. This image was captured using infrared light 
in a darkened planetarium. 

Due to a limited number of interviewers and HMDs, the students were usually tested in smaller groups. To 
begin the testing, each student was given a pre-test interview based on a series of images (as shown in Appendix A). 
Instructions for those doing the interviews are also detailed in Appendix A. The interviews were done entirely in 
ASL and were videotaped for later analysis. The subjects were asked a number of questions about the astronomical 
objects that they would soon see in the show. The interviewers remained careful not to use any of the classifiers used 
in the show during this interview and other interviews in order to avoid contaminating the data. We wanted to make 
sure that the students saw the classifier from the show, not from another source. After the interviews each small 
group was taken to the planetarium to watch the show. Groups using the HMDs viewed the show separately from 
groups viewing the ASL sound track on the dome. After the show each student was again interviewed using the 
same set of questions as a post-test. This session was again videotaped. Cameras for all interviews were positioned 
to allow us to view sign language used by both the interviewers and the subjects. Cameras were also used to record 
interactions between the subject and the HMD hardware. The videos were later translated into English and coded 
both for verbal content and subject actions by both ASL-speaking and English-speaking investigators. 

Members of the research team analyzed each interview to count the number of times the subjects used 
specific classifiers to answer the questions. Although the students would know the use of the classifier before the 
planetarium show, they likely would not connect it to astronomical objects before viewing the show. If after the 
show they had adopted a classifier to explain an astronomical object, then we would know the connection came from 
viewing the ASL “sound track” as part of the show. The questions were designed so that the answers likely included 
a specific classifier used by the narrator in the ASL narration of the video. Questions were asked using pictures of 
objects for which we intended the subject to use classifiers. This way, the interviewer did not use the classifier we 
were trying to observe. All the data is collected into Table 2 and is sorted into numerical order of the number of 
classifiers used. 
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Table 2. Use of ASL classifiers by subjects in identical pre- and post-tests. Those labeled HMD watched the show through the 
Head-Mounted Display. Those labeled Dome watched the show with and ASL narration projected onto the dome surface. 



Pre-test 

Post-test 


Pre-test 

Post-test 

HMD #1 

0 

0 

Dome #1 

0 

0 

HMD #2 

0 

1 

Dome #2 

0 

1 

HMD #3 

0 

3 

Dome #3 

0 

1 

HMD #4 

0 

3 

Dome #4 

0 

2 

HMD #5 

0 

3 

Dome #5 

0 

3 

HMD #6 

0 

4 

Dome #6 

0 

3 

HMD #7 

0 

5 

Dome #7 

0 

4 

HMD #8 

0 

5 

Dome #8 

0 

4 




Dome #9 

0 

7 

HMD Avg. 

0 

3.0 

Dome Avg. 

0 

2.8 


DISCUSSION 

Looking at the data given in Table 2, we must first acknowledge that this is a small data set. However, the 
data is sufficient to test the feasibility of using an HMD to deliver information in the form of an ASL “sound track.” 
An examination of the values in Table 2 clearly shows that none of the students used a classifier from the show to 
answer the pre-test interview questions. However, after the show we see that 15 of the 17 students used at least one 
of the classifiers from the show, which accounts for nearly 90% of the test subjects. This result is encouraging by 
itself. 


Perhaps more encouraging is that fact that the HMDs performed at least as well as projecting the ASL track 
directly on the dome surface. From a traditional t-test one would say that the mean, mode, and median for the two 
samples were identical. However, if we look at a box-and-whisker plot for the two runs of data (see Figure 5), we 
see that the students who used the HMD actually performed slightly better than students who viewed ASL projected 
directly onto the dome. For the group using the HMDs, we find that the middle 50% of the subjects were in the 
range from 2 to 4.5, while those watching the signer on the dome ranged from 1 to 4. Although the one score of 7 
cannot be considered an outlier by any statistical test, it still influences the test results. Even with this in mind, it 
remains clear that in both cases the students adapted ASL classifiers from the show to describe astronomical objects 
on the post-test. Since the HMD is at least as good as projecting the ASL on the dome, we feel it is a viable tool that 
needs to be tested further as both an education method and as an accommodation method. Remember, projecting the 
signer onto the dome would not normally be considered an appropriate accommodation for a mixed hearing/deaf 
audience since it would likely interfere with the learning of the hearing audience. 


8 
7 
6 
5 
4 
3 
2 
1 
0 

Figure 5. Number of Classifiers Used for Head-Mounted Displays and for Direct Project onto Dome after Planetarium Show 
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Providing accommodation for a deaf audience has historically come down to one of two options: Either put 
an ASL signer on the floor, or provide a captioning system. Both of these options have issues that can impact both 
the deaf audience member(s) and hearing audience members. As was detailed earlier, captioning presents challenges 
for those whose first language is ASL, not English. Often their reading skills are not up to the level required to keep 
pace with most modern planetarium shows. For either educational or accommodation purposes, captions often have 
to be shortened, simplified, or the show must be run at a slower rate. Since the vast majority of deaf children use 
ASL as their primary language, it would be far superior to deliver content in that native language. The translation to 
ASL should not be viewed differently from translating a show to any other language. This removes the need to 
shorten or simplify the information being delivered. 

The primary purpose of this round of testing was to test the feasibility of delivering an ASL “sound track” 
during a planetarium show, thus providing both a visual experience and the accompanying narration in the native 
language of the students. Would the students be able to see and understand the ASL track? In other words, would 
the students be able to watch the show, adopt ASL classifiers used in the narration of the show, and then use those 
classifiers to describe events from the show? We tested two possible accommodation approaches for delivery of an 
ASL sound track. First, we projected the ASL signer directly on the dome surface. Having a signer on the dome 
closer to the visuals would be considered an improvement over the traditional signer on the floor, as work on split 
attention has shown. While this approach would be useful for an entirely deaf audience, it would be completely 
inappropriate for a mixed audience. Second, we delivered an ASL narration through an HMD, which allowed the 
ASL signer to track with the head motion of the student. The ability to move the signer to any location on the dome 
puts the signer even closer to the object or event being described. The HMD also removes almost any impact on 
other audience members. 

We found that for both on dome and HMD “sound tracks” the majority (approximately 90%) of students 
adopted at least one classifier used during the show. This demonstrates that the students were able to see clearly the 
ASL track and associate classifiers they already knew with a new object or event. While acknowledging that the 
sample sizes are small, we feel that it is clearly demonstrated that the ASL track delivered through the HMD is at 
least as good as the same track projected directly on the dome. Since the HMD solution would be better for a mixed 
audience, this result is very promising. Although at this point we are not claiming learning as an outcome of this 
study, we feel that the results are encouraging enough to pursue a much larger study of using HMD to deliver ASL 
as part of planetarium shows or in other challenging environments. With a larger sample, learning gains could be 
examined in more detail. 

With a successful test for the HMDs, many environments beyond a planetarium exist where testing could 
be conducted. We need to try the HMDs in additional authentic settings and measure learning gains in comparison 
to traditional methods as well as “best practice” methods. Finally, we would like to note that the kids who watched 
the show with the HMD uniformly reported positive experiences. There were times when testing became difficult 
since everyone wanted to wear the HMDs. 
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APPENDIX A 

Preliminaries: 

1 . Obtain consent. With JMS the parental consent will be done ahead. 

2. Get assent from children for pre/post-test. 

a. Explain Study: An HMD is a small computer screen that you wear on your head, and only the 
person wearing it can see. We want to see if they work to let you see an interpreter while you are 
watching a planetarium show, since an interpreter can’t very well stand up in the sky where the 
video is. We also want to know what kind of HMD works best, so we’re going to have you try all 
of the different styles. 

Avoid telling how wonderful/cool we think this is. 

b. Explain what we are doing: In order to see how well the HMDs work, we are going to ask you a 
few questions now and the same questions after you’ve watched the planetarium show. We are 
testing the HMDs, not you. So, we need to know what you already know before you watch the 
planetarium show, and what you know later. If you don’t know the answer, just say so. 

c. Explain Rights: You can stop any time if you decide you don’t want to do this anymore. 

d. Get consent/assent: Ask the kids if they want to take the test. 

You will be showing the kids the pictures and asking them the questions USING THE PICTURES. Their answers are 

in ASL. This is not an English vocabulary test; we are trying to see if they can understand ASL narration over an 

HMD. So, I have tried to include images for unfamiliar words like geyser. Please use the pictures rather than using 

ASL classifiers because we want to see if they learn the CLs, not if they can copy them from you. 
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ID_ Date_ Facilitator_ 

1. During a Lunar Eclipse the moon gets dim and red. What are the positions of the Earth, Sun, and Moon when a 
Lunar eclipse occurs? 





2. This is a picture of a (finger spell) comet. The arrow points to the comets tail. How does a comet’s tail form? 
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3. This is a picture of Mercury, and the arrows point to spots named (finger spell) craters. What causes (finger spell) 
craters like the ones in the picture of Mercury? 



4. These are pictures of (finger spell) volcanos and geysers. Are these things present on any planets or moons 
besides Earth? If so, can you say where? 



5. This is a picture of the planet Saturn. Describe what makes up this ring. 



Copyright by author(s); CC-BY 


15 


The Clute Institute 













Journal of Astronomy & Earth Sciences Education - June 2015 


Volume 2, Number 1 


3. Get assent from children for HMD test. 

a. Explain what we are doing: 

A group: We have 3 different headsets. Each one of you is going to watch the planetarium show 
wearing a headset. When you are done, we’re going to ask you the astronomy questions over again 
and ask you what you thought of the HMDs. 

B group: We have 3 different trailers and 3 different headsets. We will rotate the equipment from 
one person to the next. So, each of you and each tech crew gets to try a couple headsets. We will 
have the tech crew evaluate the setup difficulty so we can reach conclusions on difficulty in setup. 
And each of you can say what you liked / didn't like about each of the headsets. This allows us to 
evaluate what type of technology people prefer. This also allows you to get used to the HMDs 
before we try to use it for a whole show. We’re also going to let you watch a show with the ASL 
up on the normal screen. We’re going to ask you the astronomy questions over again and ask you 
what you thought of the HMDs. 

b. Explain Risks: If it’s uncomfortable or if it makes your eyes or head hurt, we need to know that. 
Make sure you tell us right away. 

c. Explain Rights: You can stop any time if you decide you don’t want to do this anymore. 

d. Get consent/assent: Ask the kids if they want to go to the planetarium. 
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